Usability Testing of e-Learning in Practice: First Experiences and Lessons Learned

As part of National College of Ireland, we have recently opened the National e-Learning Laboratory (NELL), a research facility specialising in usability testing. Since its opening in December 2008, researchers in NELL have been involved in a large number of usability projects with a variety of different clients. As testing has developed, we have been investigating various methodologies of how to evaluate products being tested and how to best describe user experience. The aim of this short paper is to present our first experiences of usability testing; to describe our research methodology and to discuss lessons learned. We look at some common themes that have emerged from the usability research projects; such as the extent to which user expectations are guided by previous experience and how, when the system did not conform to prior expectation, this had a considerable negative impact on ease of use and satisfaction. We also consider some lessons we have learned as researchers while undertaking the testing, such as the usefulness of user personas and scenarios when designing tests and the importance of making user tasks non prescriptive. We conclude by looking at the issues concerning usability testing specifically related to e-learning, where surprisingly a coherent and widely used usability framework is not yet available. We consider what is necessary for the future development of such a framework.